Bayesian Regularization in Constructive Neural Networks

In this paper, we study the incorporation of Bayesian reg-ularization into constructive neural networks. The degree of regulariza-tion is automatically controlled in the Bayesian inference framework and hence does not require manual setting. Simulation shows that regular-ization, with input training using a full Bayesian approach, produces networks with better generalization performance and lower susceptibility to over-tting as the network size increases. Regularization with input training under MacKay's evidence framework, however, does not produce signiicant improvement on the problems tested.